CtsJOl^OO^ PODS' tifctPi^^ 



(12) INTERNATIONAL APPUCATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



CORRECTED VERSION 



(19) World InteUectual Property 
Organization 
Internationa] Bniean 

(43) International Publication Date 
9 December 2004 (09.122004) 




PCT 



(10) International Publication Number 

wo 2004/107688 Al 



(51) internatioiial Patent Classification^: H04L 12/58 

H04Q 7/22 

(21) International Application Number: 

PCr/EP2004A)50874 



25 May 2004 (25.05.2004) 
English 
English 



(22) International Filing Date: 

(25) Filing Language: 

(26) Publication Language: 

(30) Priority Data: 

03138197.9 29 May 2003 (29.05.2003) CN 

(71) Applicant {for all designaied States except US): JQVTER- 
1 NATIONAL BUSINESS MACHINES CORPORA- 
i TION [US/US]; New Orchard Road, Armonk, New York 
10504 (US). 

(71) Applicant (for MG only): IBM UNITED KINGDOM 
LIMITED [GB/GB]; PO Box 41, North Haibour, 
Portsmouth Hampshire P06 3 AU (GB). 



(72) Inventors; and 

(75) Inventors^Applicants (fijr US only): UN, Hong 
[CN/CN]; 3d building Yangguan, Xiaoying west street, 
Beijing 100085 (CN). SONG, Song [CN/CN]; No 143. 
East 9 Building, Northern Jiaotong University, Beijing 
100044 (CN). SUN, Pd fCN/CN]; Room 403# Building 
30#, Yangguang Nanli, (JingHe, Haidian District, Beijing 
100085 (CN). SHEN, Jun [CN/CN]; Room 2-801, Build- 
ing 27, 5th Block, Tian Tong Yuan, Chang Ping District, 
Beijing 102209 (CN). 

(74) Agent: LITHERLAND, David, Peter; IBM United 
Kingdom Limited, Intellectual Property Law, Hursley 
Park, Winchester Hampshire S021 2JN (GB). 

(81) Designated States (unless otherwise indicated, for every 
kind of national protection available): AE, AG, AL, AM, 
AT, AU, AZ, BA, BB, BG, BR, BW, BY, BZ, CA. CTI, Cn[ 
CO, CR, CU, CZ, DE, DK, DM, DZ, EC, EE. EG, ES H 
GB, GD, GE, GH. GM, HR, HU. ID. IL, IN, TS, JP, Ke! 
KG. KP. KR, KZ. LC. LK. LR. LS. LT, LU. LV, MA. MD. 

[Continued on next page] 



= (54) nUe: METHOD AND SYSTEM FOR MULllMEDIA MESSAGE SERVICE COMMUNICATON 



I Intera ctive Multimedia MessagefPl) 



, , _ iPTesentatio n P(»tjon(P2) 



■ . V o . .f ^*^g^tatioii stnicture ff3^ 
[sAerafe'lfeijp^^ 



00 
00 




^ Actions list (P 8) 



ivc Message dispIaycd(P6/nf 
" Action Eventr"" 



Content Presentation r^f m^rPIO) 




New Message Content(P 1 1 ) 



2 Sl^ An interactive method, a user terminal and a communication system am provided for multimedia message services 
§ ;St^T*\ ! ^hexcin one or moie conlrollers an= embedded L the display^ or played mdfSmesSc 



wo 2004/107688 Al IBMiiMiiiilimiiliHiMllllMim 



MG. MK, MN, MW, MX, MZ, NA. NI, NO, NZ, OM, PG 
PH. PL. PT. RO. RU. SC. SD. SE. SO. SK. SL. SY. TJ. Tm! 
m TR. TT, TZ, UA, UG. US. UZ. VC. VN, YU. ZA. ZM. 



(84) Des^nated States (unless otherwise indicated, for every 
kind of regional protection available): ARIPO (B W, GH, 
GM, KE, LS. MW, MZ, NA. SD, SL, SZ, TZ, UG,' ZM^ 
ZW), Eurasian (AM, AZ, BY, KG, KZ, MD. RU, TJ, TM)i 
European (AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES. Fl, 
FR, GB, GR, HU, IE. IT, LU, MC, NL. PL, PT. RO. SE, SI 
SK, TR), OAPI (BF, BJ, CF. CG. CI. CH GA. GN. Go! 
GW, ML. MR, NE, SN. TD. TG). 



Published: 

with international search report 

(48) Date of publication of this corrected version: 

6 October 2005 

(15) Information about Correction: 

see PCT Gazette No. 40/2005 of 6 October 2005. Section U 

For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes ondAbbreviaiions" appearing at the begin- 
ning of each regular issue of the PCT Gazette, 



wo 2004/107688 



PCT/EP2004/050874 



MPi®K'^3^c:.x;:~:'"^&M0V zoos 



METeOD AND SYSTEM FOR MULTIMEDIA MESSAGE SERVICE 

COMMUNICATION 

Technical field 

•Die present invention generally relates to the oomimmicaiion tedmolQgy. and in particular 
■> toanintaactivemethod,auserlemiinaiandacornni^^ 
services. 

Background Art 

As an atliactive valiie-added sendee. Ite sh(« niessage sema 
becotning another iinptwant service area in moh^ 

as a leading function. A wide range of different value-added data services are based on this kind of 
mobile datacommunication means of short message such as mobDe phone bank, mobilephone 
secuiities. infonnation on-demand, wireless e-inail. wireless data communicalion and fex etc. All 
these are referred to as "short niessage service" (or SMS) in general. 

At present. SMS is a fairiy basic data service that enables a wiidess subscriber to send 
simple text messages of up to 1 60 characteis at die present SMS has, as a result of the 
development in communicalion. been devetoped fiom a simple notification means to a vensalile 
platform for value-added services. 

Taking fig.l for example, where a conventmnal solution is shown for SMS based 
value-added services adopted for wireless subscribe.^ For example.if auser wantstoaskfora 
current stock quote of IBM via SMS. he or she has to find the IBM slock quote service code (SQ) 
and the service provider access number (Le. mes«^e center number. e.g. 1 23456), and input the 
command fomiat 0^. the service code plus the company name: SQ IBM), then sends this to a 
message center to request the desired information about the stock quote of IBM. Upon receivi,^ a 
request from a subscriber, the message center (or network server) provides the content requested 
by the subscnT^ in response. Aldiough it is very tn>ublesome to remember and 
relating to this kind of requests, the SMS based value-added savi« gives users the convenience to 
access and to acquire information. 

Because the cunent short message service supports mainly relatively short texts, the future 
development in contents of the short messages win be focused on multimedia contents. Tht most 
prominent feature of this kind of richer short message service, however, is to s^ 
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messages to pass on video cUps. pictures, audio samples and text Multimedia message service 
(MMS) has been proposed in consequence of the technical upgrades in the short message service. 

MMS is the natural evolution of SMS. TTie messaging standard for MMS is set up by two 
Qigamzations. WAP (Wireless Application ftotocol) Forum and 3GPP (3^ Genaation Partnership 
■ Project). HieiefoiB MMS is designed to operate at upper levels of the WAP protocol, without 
limitation to a oeitain art of transmit 

and the general packed radio service (GPRS) data communication. MMS cuneotly adopts 
WAP-push technique, which is a store & transmit function similar to that of SMS, and therefore the 
cunratMMS technique remains yetastoreAransinit one. TOs means Aattf^ 
handset.arBcipient win not iBcdve it directly. Instead of that. Ae message is received in advance 
by a multimedia message center of &t network in which the subscriber resides, and then the 
multimedia mesa^ center sends a notification to the recipient infonning that the recipient should 

download the message fiom the multimedia message cmter. A similar inultitnedia message c 
exists also in SMS systems, but as all the messages to be sen t are text ones, it is iBl^ 
operatioa Whereas by MMS, the operation in the message center is relatively complex as it will 
not rely onaoertain network ofacertain operator, so if, for exan5)le. the red^^ 

slower network, or his/her handset- s screen has a dififerem size, the equipment at Ae ope^ 
needs to be able to detect the situations and to configure a conespondii^ message format 

MMS is a new global message communication standard, the most prominent feature of 
whichisihesuppoitingafmultimediaappfications. The multimedia message enables contents and 
pieces of information wifli a M range of functions, including images, audio infomiation, video 
information, data and text, to be transferred, and video cUps, pictures, voice and text, supported by 
a GPRS network, to be transferred, by taking advantage of WAP protocol as a carrier. When the 
MMS specification was constituted by the standard organization it was once conceived diat MMS 
should be an appKcations bearing platform serving not only as a store-forward center for messages 
but perfonnipg various enriched applications. Through a mobile terminal supported by MMS a user 
should enjoy the same content services and experiences, as he or she would get through the 
classical Internet As MMS can support a wide range of data format standards, such as image 
fomwt audio format animation format standard, that entirely the same experience as get fiom the 
internet is therefor desired to be available for user by MMS, or even multimedia data stream 
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siqjport is expected, when, in future band wi^ 

aqpression capability and enrich messa^ contmts. MMS promises a dramatic increase in 
messaging capabilities (hat win enrich user experience and create a major new source of revenue for 
networic operatore as well as content and service providers. However, a primary shortcoming or 
5 in¥erfectionofthecunratMMSslandaniisfliatMMSisstiUinasitualionof"sin^ 
communication", lhat is to say, most of the short message services available at present are 
unidiiBcdonal, generally do notprovide interaction between conient sources or providers and end 
users. Although some features of interaction between end users and servers (or content and/or 
service providers) have been proposed, involved only in those features that a mobile subscriber 
10 send a message with a mobile terminal to reply a infomrationre^ 

demands for infonnatkm interaction fiom users, and this fact limits die devdopment of die 
value-added MMS sovices. 

MMS is the natural evolution of SMS. Because MMS delivere much richer content than 
SMS, MMS provides a fiiendly and vivid mediod for expression. However, the service model 

inherits the MMS, and users have to compile die request messiige by dicmselvBS. And 
currently most of MMS plications only focus on how to transmit and present images, ring tones 
and text, but do not talk about how to make fall use of MMS characteristics to enhance the user 
experience in MMS value added services. Especially in most of MMS applications, attention is 
scarcely paid to provide or improve interaction services between end usere and servers (e.g. MMS 
providing servers) and to aim at a MMS available on a server, and a more meaningful local 
interaction at users mobile taminal is impossible. For example, an associated operation on 
individual infonnation items in a received multimedia message is not realizable. The contents of 

multimedia message diat can be dispkyed on a user temunal arc liinited and, the user needs eithff 
by repeating many times of operations or operating on the message displayed to get a further 
information, or by compiling request message and sending the noessage thus composed to the server 
to get more infoiniation. Needless to say, it is not only diat the message contents that a user can get 

are linrited, but that the procedure a user must lake is boring and troublesome. aD of these le^ 
the experiences a user get in multimedia messages. 

As it is shown in Figure 2, where a MMS structure is simply iUustrated, a multimedia 
message consists of MMS headers and a message body. Hie MMS headers contain such 
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M(mnatm on IKW to liansfo Ibe mulii^ 

address, destination address and etc. n« message body consists of several parts including 
multimedia objects, such as image Qpeg fomiat). text (plain format), audio (wav fonnai), eadi in 
separate part, as well as the optional presentation part The presentation part contains instmctions 
5 toe^lainhowtherndtimediacontentshouWbercnderedlnthepriorart 

alternatives of the compuiBrpiesenlalion language, which deals with how to dfc?)lay presentations. 

One of the presentation langusiges mostfy used Iqr those si^ 
Multimedia Int^onDmguage (SMIL). SMIL is a common deploymertf for MMS presentation, 
ft is an important way to inlj^grate multimedia into Web content SMIL 
D language (XML) based language The XML language aUows to describe timing indicated by 

multimedia, to associate supper-links with multin^a objects and to define the layout presented on 
the screea SMIL is regarded as a way for enriching of the current text mess^ based transmission 
technologies. SMDL consists of a set of modules that define the semantics and syntax for certain 
areas of fimctionaUty. These modules are layout module, timing and synchronization module and 
animation module. SMIL merely describes how to show a message on a screen, but it does not 
provide any interactive features forendusers.SotheSMIL needs tobe extended to support 
interactive presentation, because interactive messages can provide good user experience to 
popularize the MMS technology to wider appHcation areas. 

Another presentation language mostiy used by those skilled in the art is Hypertext Markup 
I^guage (HTML). Ihe HIML is the most popular language supporting interactive presentation in 

W^appHcation^ But the HTML fomi(aformpresentingasamm document) camiot be «^ 
in MMS. because the HIML form cannot separate the content from the presentation, in other 

words, in a HTML form the preser,tation is often mixed with the data. As matter of fiua^ 

so many MMS clients with quite different capabilities, and fl« sq«ration between the content and 

die presentation would make it possftle to support different MMS clients. This is a basic 

requirement for aMMS mess^gppresentation language. Furthemrore. the protocol adopted in the 

networitservk^program. for example ihecommunication model of Hypert^ 

(HTML), is requestAespbnse. This model leads to more lound tiips between the service and the 

client's mobile temrinal MMS is a message system, which mns over wid^area wireless network It 

is not a real-time message. If the request/re55K)nse model could be implemented, the round-trip will 
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bring huge latency and reduce the pafonnanoe of the whole system. What is needed is to display a 
ridier <^>naniic fimn in an opdnnim way on a MMS client's termi> )al 
Disclosure of the Invention 

In view of the above piobleim in the prior art the aini of flie ptesert invention ihere^ 
to provide an interactive method fhr mnltinnvtia mfssage service;:. 

An object one or moiB embodiments of die pesoA invention is to realize an intoactive 
multimedia communicatton between subscribers and a MMS back end system, for example a 
multimedia messs^gp providing server, preferably in response to 

fiom a server a response message is generated automatically to be sent back to the server without a 
multq)le round-trips between the user temnnal and the server, and according to the present 
invention a flexible, richer and more convenient local interaction at user terminal is possi-ble. 

Another object of one or more embodiments of the present invention is to provide an 
optimized user interface of the MMS message for users. To this end one or more interactive 
controUers arc embedded into the multimedia message, the user performs the local interaction of tiie 

user terminal as wen as the interaction between the user terminal and the SH^ 
multimedia message sendii^ server). 

Yet anoflier object of an embodiment of the present invention is to create a dynamic Web 
fomi. which is delayed on the user terminal, in XFomis language, preferably a plurality of dynamic 

fomisinduded in an sfagle message presentingpagetoprovide richer interactive presentati^^ 
more optimized multimedia messages for the user, to enhance his or her experience. 

According to the present invention an interactive mefiiod for multimedia message services 
is provided, comprising steps of: receiving a multimedia message fiom a server, genetatii^ a p^ge 

disphying or playing the multimedia message at a user leraunal. wherein one or more cont^ 

embedded into thediq)layedorplayedmultimediamess^andassociationsbetween said 

controDers arc defined; and ttiggering at least one action event and performing a corre!^ 
action in response to die operations of one or more controllers. 

In one embodiment, said triggered action event is to send back a response message to flie 
server which renders said multimedia message; and in response to the operation of said controllers, 
automatically generating a response message for request to submit an order. 
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In anothCT«nbodiment, said triggered action evoit is to modify the contait of said 
multimedia message; and in le^nse to tbe opoalion of said controllers, displaying or playing said 
modified content of said multimedia message. 

In another embodinient, the slq)ofgenCTaling the pa^ for displaying or pl^^ 
multinaedia messa^ fuifher includes : parsing the leceived multimedia message to obtain the 
presentation structure of said multimedia message; and gmerating a data model used in flie page for 

displaying or playing said multimedia messages based on said multimedia message presentation 
structure. 

According to anoflia aspect of the present invention, a user terminal for multimedia 
message service is provided, adopted to receive multimedia message firom a sever and process said 
multimedia mess^ said user tenninal comprises: a device for receiving multimedia message bom a 
saver, a device for g^erating a page for displaying or playing said multimedia message; a device 
for embedding one or more controllers, between which relationships arc defined, into said 
displayed or played multimedia message; and a device for trigging at least one action event in 
leqwnse to an opOTtion of said one or more controllers, and for mqjlementing flie corresponding 
action. 

According to y^ another aspect of the piesrat invention, a OMnmunication system for 
multimedia mess^ service is provide, said system comprising: one or more user terminals; one or 
more serv^ for sendir^g a multimedia messzge to said user terminals, characterized in that said user 
tenninal comprises: a device for receiving a multimedia message fiom a server, a device for 
genoating a page for displaying or playing said multimedia message; a device for embedding one or 
more controUers, between which relationships are defined, into said di5)Iayed or played multimedia 
message; and a device for trigging at least one action event in response to an operation of said one 
or more controllers, and for performing the corresponding actioa 

The inlOTctive s^jproadi for the multiniedia message service aoc^ 
invention, tfie usar terminal and corresponding communication systan as weU as the corresponding 
con5)uter program provide a method for generating the value-added message based message for 
niultimedia message irovidas, a fiiendly user interface, which enables a more vivid dynamic form 
to be presented in an optimum manner, for the MMS user, and a more convenient way to send a 
service request for the MMS user. According to flie present invention the usct can access more 



wo 2004/107688 



PCT/EP2004/050874 



7 

infonnalioii while flie round-trips between the sffver and the usa- mobile tramnal arc reduced 
without taking up too many systan resources or causing latency. According to the present method, 
the performance of the message service system is improved. 
Brief Description of the Drawings 
5 Embodiments of the invration will now be described, by way of example only, and with 

reference to the following drawings: 

Figure 1 schemalicaUy shows a SMS interface presented on a user lemrinal; 
Rgure 2 schffliatically shows flie message structure of conventional mnlHmf^jia message; 
Figure 3 schematically shows a Mode diagram of a MMS communication system 
1 0 en5)loying the mefliod of an embodiment of the present invration; 

Rgure 4 schematically shows the intOTCtive pioced ure f or the multimedia messagp scavioe 
according to an embodiment of flie present invention; 

Figure 5 schematicaUy shows the flow diart of the procedures of the interactive method 
for the multimedia message service according to an embodiment of the present invention; 

Rgure 6 shovirs logic models of flie interactive controller according to an embodiment of 
the present inviention; 

Rgure 7 shows one of tfie controllers according to an embodim^t of flie present 

invention; 

Rgure 8 sdiematically shows an example of flie intCTactive multimedia message service 
iix?>lemCTted according to an embodiment of the piesent invention; 

figure 9 schematicaUy shows a data model of flie multimedia mess^ according to an 
embodiment of flie piesent invention; 

figure 1 0 sdiematically shows a data model of an exen?>lary multimedia messa^ 
illustrated in a metfiod for realizing ttie interactive approach for multimedia message service; and 
Rgure 1 1 schematicany shows a functional digram iUustratirig an middleware process 
acconling to flie present mefliod. 
Mode(s) for carryir^ out the Invention 

Hereafter a detaited description will be given to embodiments of flie present invention wifli 
leferenoe to accompanying drawings. It should be understood fliat ttie following description enables 
fliose skilled in flie art to cany out present invention Various chaiigps and modifications will 
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become apparent to those skilled in the art, and the teachings of the present invention can be 
appUed to olher embodiments. The present invention is therefore not intended to be limited to die 
exeasplary anbodiments described bdow. 

Reference is now taken to Figure 3 which schematicany shows a block diagram of a 
5 MMS communication system. The MMS commimcalionsysto 

interactive nmhimedia message, leahzing oommumcation between a usa- mobile lominal and a 
sen«r,fijreicampleamultimedia message server. A MMS commimicaiion system generally 
comprises of three main parts: a MMS value-added service server 300. a communication networic 
and MMS center (MMSQ 200. and one or more MMS terminals 100. The value-added service 
10 sen«r 300 provides arid sends mdtimedia message, and can recdve request in 

the MMS temiinal 100. Pteferably MMS value-added service serve 300 may comprise a 
communicaiion module 301. a message parser module 302, and one or more service module 303. 
The communication module 301 is adopted to process various possible communication protocols, 
for example HTTP or SMTP. The message parser module 302 parses incoming request message, 
for example, fiom the MMS temiinal 100 to derive die format and content of message to be 
presented, according to the association information contained in die message header, fiom die 
received message, and then passes the request message to a corresponding service modules 303. 
The service modules 303 are used to process the service logic. The communication network and 
MMSC 200 can be provided by suitable communication operator. The MMS terminal 1 00 may be 
any oomnmnication device, which allows multimedia message communication, eitiier an individual 
multimedia communication device or various other communication devices which have die capability 
of canying out multimedia message communication, and die conespondiiig hardware structure, die 
MMS temMnal includes but is not limited to a mobile teksphone; a personal digital assistant (PDA) 
and etc. In general, the MMS tenminal 100 comprises a MMS module 1 10 for carrying out die 
multimedia usex appHcation prpgram on Ok MMS terminal 100. Through nmning the multimedia 
user application program on the MMS terminal 100. die local interaction on die MMS terminal 100 
and die interaction between MMS terminal 100 and die server 300 are realized. Pteferably, die 
MMS tenninal 100 includes at least four components: a communication module 1 11 . a parser 
module 1 12; an interactive module 1 13, and a storage module 1 14. Wherein die communication 
module 1 1 1 mainly process various MMS communication protocols so as to communicate widi die 
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server 300 via theMMSC 200. The parser module 112 parees the multimedia message received 
from the server, reads out the information in respect to the message format to be presented on the 
MMS tenninal 1 00 and die related infonoMtion, so as to cany out conespondiqg process on flie 
message of various presentation formaL Should, fijr example the lecdved message format be 
pictorial a picture information is displayed accordbg to the picture message format; an audio one. 
an audio message is played; and a textual one, the content of a text is displayed. Hie interactive 
nrodule 113 caniesoutthe associated operations corieqwndingvwth individual contra 
Ae multiniedia message fomi, according to the relalionsh^ between the media objects, wMch have 
been parsed from the received multimedia message, to realize the local interaction on die MMS 
terminal 100 and the interaction between MMS teiminal 100 and the server 300. The storage 
module 114 is used to store the received message and involved process results therewith. 

Now refoence is taken to Figure 4. Egure 4 schematicaDy shows the interactive 
procedure for the multimedia niessage service according to an embodiinem of the pr^ 
Ristly, in procedure 1 30 the server 300 sends an interactive multimedia message to the MMS 
temainal 100 via a communication network and a MMSC 200. The mess^ is. for example, sent 
once and may be pennanentiy stored into the MMS terarinal 100. 

In procedure 120, a process of a local interaction on the MMS terminal 100 is shown. 
TlK local interaction process is a process tiat deab mainly with user tqjerations, 
user perfonns operations, including adding, deleting, modilying, entering, selecting, searching and 
etc., on various forms provided on die user interfece. Of course die user can either choose one or 
more from said operations, or perform none of them. 

Depending on the requirement, an interaction between a MMS temrinal and a server is 
also realizable. In procedure 131. a request message comprising request message inputted or 
chosen by the user, is sent from MMS temunals 100 to service server. Reftrably, a response 
message with the desired service is sent bade (procedure 132) to the server 300. 

Hereafter, the interactive qpoach for multimedia message service is fiirflier explained in 
details with refenraioe to Figure 5. 

Figure 5 SChematicaUy shows the flow chart of the procedures of the interactive approach 
for the multimedia message service of an embodiment of the present invention. TTie procedure steps 
iflustraled in the flow chart are mainly peifomied ttaough a MMS module assembly 110 in the 
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above^nenti(medMMSienniiial 100, that is ihatfteimsentembotUment is realized through 
perfomiing the user appHcalion program, which runs on the MMS module assembly 1 10, in the 
connmunicalion module 111. the parser module 112 and the interactive module 113. Of coune die 
descr5)tion of the embodiment is only given in an illustrative manner, and the piesent invention is not 
5 linuted to die Structures shown in the accompanying drawings. 

Now reftreiMje is taken to figure 5. In view of the multiniedia message (PI) 
multimedia inessage server 300, in step F2. the mdtimedia message is recd^ 
temainal 100 via MMSC200. Rrsdy, parser module 1 12 parses die presentation portion (P2) of the 
message out of the receh«d message data. Hie presentation portion (P2) is fiequenfly involved in 
) the mess^ content to be presented. Next, in connection with die presentation portion (P2) of the 
messagp, die presentation portion is further parsed in step F2, tiiat is, an analysis is earned out in 
respect of the description about how to display the message. Tlie presentation structure (P3) of die 
presented message, acquired dierefrom, is die related message format to be presented on die 
MMS teminaL Besides, while parsing of die presentation portion of die message is carried out, die 
nJes of relationship and activities between contronets is parsed and dieo die relationdiip and 
activities are embedded in an action fbrai (P8). Based on die parsed presentation structure CP3). in 
step F3, a data model (P4) of die presentation messi^ or display message is created according to 
diflwMt message fimiiats, sutA as a text fbna^ 

prcfena)lydiq)IayedinaMarimp language. In view of die created data model 
or playing message, corresponding data are loaded and a message page (P5) is created in step F4. 
In step F5. die created message page is displayed or played on die MMS temiinal 100. In view of 
die presented multimedia message die user performs operation on it, and in step F6 in response to a 
user operation an action event (P7) is created. And in step F8, an action listener is used to monitor 
die action event (P7), die action event is processed and a coiresponding action is generated. The 
action listener is. in step 7, generated based on an action list defined for regulation regulations, 
resulted fium die parsing of die presentation portion petfbnned instep2Instq)9.inviewoftiie 
action (P7) monitored in step F8. an event-processing program is activated to process die user's 
action. Hie monitored actions ate. for example, divided into two kinds, one is to change die 
contents (PIO) on die message page on die MMS terminal. For tiiis kind of action, step F5 is 
repealed, diat is, die action is processed on die basis of die action rules defined in die action Ust. so 
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as to display or play the changed mess^ pagp on the MMS tenninal 100. The other is involved in 
the interaction with Ae service, in step m on the basis of the new message 
data, if the event is an event involved in message deUvery. a mess^e is generated in accordance 
with (he inessage noodel and the user's preference and then the generated rn^ 



300. 



According to an embodimeat of the present invention, when the MMS presentation model 
is created one or more interactive coDtiollers are provided on a multimedia message presentation 
intefeoe in which nimber of forms aiB provided The inte^ 

buttonfoanordfffimn,aiadiobutton.achedcbox.atexutcxtfield,alistte^^ 
and etc. Each of the controllers can be designated a name. When a fomi is submta^ 
cuntentvalueof a specific controller can be sent together with the fomi By embedding controllers 

into the multimedia message, the user can express his or her preference to service 
program by means of these interactive controllers, to provide an interactive sohition. 

Now the nefereoce is made to figure 6. which shows logic modules of the interactive 
controllers according to an embodiment of the present invention. 

Tlie present embodinKnt supports multimedia iterns throughout extendi^ 
controller. 11« supported multimedia items mjqr be image, audio and video. The ite^ 
button, the diedc box, the Hst box and fl» option menu may be pictures, audio and vid^ 
Taken the option menu for example, the items of the option menu could be pictures. Each picture 
lepiesents an item. Take the multimedia options memi shown in Fig^ 
models of Nokia handsets, for example Nokia2110, NokiaSllO, Nokia6110, Nofcia7250 etc and 
pictures of the respective models of handsets are iUustiaied The user can. for example, dick a 

handset picture for one of the various models, and the handset of this model is displayed on the 
display screen of the user temrrinal, MMS terminal 100. The user can also move the 
display more Nokia models of handsels and pictures. 

Hereafter descriptions will be given to the reMonsh^ between the interactive controllers 
proposed by an embodiment of the present invention. Logic modufes are adopted to define 
relationships and actions. The relationship between die interactive controllers includes two kinds of 
i^onships, namely UNK and CONTAIN. For example, if a LINK relationship exists between 
two or more imeractive controllers, when one of the controller items is selected, all of others are 
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selected too. ConsequenOy. when one of Ihe controller items is displayed, all of others are 
displayed or played too. As it is shown in Kgure 6, when the model of a handset is changed bom 
Nokia 5100 to Nolda7250, for eveiy model of handsets dieir pictures and prices are changed in 
consequent respectively. CONTAIN is another relationship. For example, if the item 1 of die 
controller A contains items 1 through7of the controller B, when item 1 of the controller A is 
selected, only items 1 through 7 of the controller B can be selected. 

Accoidiflg to an embodiment of the present invention, firstly the ea^ 
extended to diq)hyaricher dynamic fomi on the user terminal, whaeinflie^^ 
controllers have certain association, preferably the relationship between controltes are defined as 
UNK iidations or CONTAIN relations, to fioUtate the user to operate on the displayed 
nuiltimedia message, such that a convenient means for the interaction between the subscriber and 

the backend systems is provided Ausercan select and input his orherprcfercnce by means of 
these interactive controUeis, and the interactive controlleis designed according to the present 
invention can automaticaDy generate a response message based on die user's choice. The user 

pushes the oixJer form submit bu tton. and a response message is generated and sent out 
example, a response message, which contains an order request Lc. an order ftam. is generated 
automaticaUy. and sent to the server. A common solution is provided to message serveisAisers to 
generate/operate on value added services by means of extending the MMS system. A vay fiieodly 
userinterfe^oftheMMSmessageisprovidedtodieuserlqrmeansofusingt^ 
controllers. When the user selects different parameters, the message is displayed with a different 

presentation structure. In this manner, the user can accomplish alocalinia^ 
with a message saver. 

The extending of the existing MMS system according to an embodiment of the present 
invention is realized by embedding XFbmis technology into MMS presentation language. The 
current MMS standard is extended to enhance the interaction capability for the MMS message. 
Ihe XFbims is extended to support die multimedia interactive controUers. The relation^ 
thecontroDersisdescribedbymeansofaXMLbaseddocurnentR)rexample,flierelation 
description is named Relationship XML (RXML). And a MMS middleware system is introduced 
to process the interactive multimedia me^a^ service. 
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XFonns is an XML-based language, which defines fonn-based interaction interfaces. 
XFonnsisaienewalof the HTML fonnoonqxjsed in them 

extendable method to contain richer and more dynamic fonrn in the HIML fila By splitting a 
amveniional Him foim into thiee parts, namdy data modd.exem0^ 
i 'I^P^telionissqwratBdtalhecontent.soamdtiplaisanow^ 
fonction is availabte.XFbnnsandXMILareaUXML based lang^ 
into SMIL XFbnns provides various kinds of interactive controlleis including a submit button, a 
radio button, a chedc box, a text, a text field, a list box, an option menu and etc. 

Tliese interactive controllers arc extended to support multimedia related objects. "Die 
prcscntembodimentsi^^mullimediaitemsthroughextendingm ihe 
multimedia objects that can be supported, for example, may be pictures, audio and video dips. The 
items of the radio button, the check box. the list box and the option menu may be pictures, audio 

and video chps. Taking the (^tion menu fir example, an item of the menu is a picture. Ek^ 
represents an item. 

•Die relationship between the interactive controllers is described in XML language. 
According to an embodiment of the present invention die relationsWps between the 
controllers are defined as two kinds of rclationshps. Hiey arc LINK and CONTAIN relations. 

Themostcommonly used relation is the LINK relation. Fbr example, if there is aLINK 
relationship between two or more interactive controlleis, when one of the controDer items is 
setected aU the other are selected too. Conscquendy. when one of the controller items is displayed 
afl die rest items arc displayed or played at die same time. CONTAIN means that, if the item 1 of 
die controUerAcomprises items 1 throughVof die controller B. when item 1 of the controller A is 
selected, only items 1 tfirough 7 of die controller B are sekx:ted. 

Hereafter the relationships between die controUeis are furflier explained by means of 
txmpks. 

Taking motorcycle shopping for an example, when a user receives a motorcycle 
promotion message (lefer to Figure 8. motorcycle promoUon interactive message) and is interested 
in buying one. Tlie user can preview the pictures and prices (LINK relationship) of different kinds 
of motorcycles simultaneously by simple select fiom the model options menu. In other words, as die 
user chooses a model of a motor die picture and die coiTCsponding price are displayed in 



wo 2004/1076W 



PCT/EP2004/050874 



14 



consequent Furthermore, if a user wants to buy a model of motorcycle, a Buggy for example, what 

he needs is only to select the model and click the order fomi submit button (to submit an order 
form) as welL 

Figure 9 schematically shows a data model of an embodiment of the present inventioa 
5 Whflein, the object 1 is included in the object 2. The object 1 and the object 3 have an aUy 
relationship. The anribute of the object 1 define those of die object 2. If the objea 1 is changed 
firan one condition to another, for example the attribute 1 dmges. the object 2 and 3 are changed 
inconsequence. 

Hgure 10 schematically diows a data model of an example of motorcycle buying. In the 
10 portion of presentation structure, the data model of the message shows five options including motor, 
riiodels, and types of die wheels, pictures, prices and order form submit buttons of bicycles. 
Generally, instance data gives actual data on the basis of a data model Two pieces of message may 

have the same data modd but diffeimt instance data. In Figure 11, the instance data K 
about motorcycle promotion. The user intet&ce defines the position, format and sire of the 
indrvidual items. 

Relationsh?) or action portion describes the relationship between the object and the action 
at the tirne an everrt takes place. Usuafly a rnessage comprises inforinalion 
objects no matter how to present the information. Relationship or action portion describes the 
internal rdationshq) between data and the internal relationship between the attributes. For example, 
one attribute is defined by other attributes or other attributes are defined by one attribute. Iherc arc 
also exienial relationships betwera the objects. 

For example, in the relationship/action portion, the relationship between motor, models, 
pictures, prices and types of the wheeU of bicycles is shown in Figure 10. Motorcycle types have a 
group of models; therefor motorcycle objects include model objects. When a type of motowycle is 
selected, models that bdorig to this type can bepresented in flie type option menu of fliis type 
(CCKTAIN relationship). Each motorcycle model correjqwndents to a respective picture. But 
there are some types of wheels available for users as options. When a user pushes the order form 
submit button, a response message is composed by the MMS client appUcation. The response 
message is ^aerated oa the basis of the push action. 
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To provide a friendly inta:action feature so as to amdi the user experiaice of the MMS 

vahie-added service, in an embodiment of the presoit invention, it is achieved by describing the 

service expression in extoaded XML language as following: 

<?xml vQrsion='1.0" ^coding^ 'lSO-8859-l"?> 
<smil xmlns= *http7/vwvv.w3.oig/2001/SM]ia^ 

xmlns: xfiB= 'h^:/Avww.w3.oisQ0Q2/01Morms*b^ 
<xfin: model id= Tannl"i> 

<xfin: submiflnfo id='*submit 1" method2= '^stxmnocalfile= •ten>juri" 

target2= http://wvyw.ibnLcom/motODromQle/' 7> 

//attention A 

Kx&ni instance xmlns="^ 
<moto> 

<motox/moto> 

<modelx/modd> 

<wheelx/wheet> 
</moto> 
<4cfiTdnstanoe> 

</ xfmrrmx fe]> 
<body> 

<par iegion='^feience"> 
orfmqptianmenu iBf=*Ynoto"!> 
<xfm: caption>L Sdea the moto</xfin:cq)tion> 
<3cfiiuitem>Buggy</x£tndtem> 
<6cfin: option> 

<xfin; optionmMiu id5='taoder^> 
<xfin: c^tion>2.Select the modd<Afm: c^tion> 
ocjfiax item>NQmMkALfinntem> 
<xfin: item>ModeI A<fxfm±ea£> 
<xfia- itea3>Modd B<f}daadtBao 
<fxfm: oplion> 

<xfm: optfonmoiu iBfc*Vheer'!> 
<xfin: cJ5)tion>3 JSelect the wheels</!xfm; caption> 
<xfiix item>Normak^cftiritem> 
<3cfin: itBm>Enhanoed</?dEaritem> 
</xfin; option > 
</pai> 

<par region=**submit'S 

• <xfin:submit name=*%ubmit" lefc'bioto" to=*^mitrb> 
<xfrn: C5^)tion>Subnjit</xfin: caption> 
<W5n: submit > 
</pai> 

//the img region to display the motor picture 
<par region='1mgR^(Mf ' duit= 'SncfefinitB'yi> 
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//attention B 

odBm optiaimfimi tDodd=" picture'* refc" picturc'b> 
<xfin: c^tion> <^tfixL-caption> 
ocfiai* ilem>buggy jpg</\fiiuten3> 
► <xteitem>buggy_ Ajpg</xftaiteD^ 

<xj5ir i(em> buggyi B.jpg <yxfinatBni> 
</jcfin; option> 

<xfiii; ofMionmrnuiefe'^rice'^ 
ocfin; c£^tion> </x&n: capiion> 
ocfin: iteai>$1250</xft2i:itmi> 
<xfirL- ilem>$O50</xfaKitmi> 
<xfin: item>$1399</xfi3utem> 
<xfin:op(iQn> 
</pan> 
</body> 
<^smil> 

The multiinedia message service presentation uses SML and Xf onn to describe the 

naessagp user interfeoe. Hie interactive contioners are presented by XFonm, When a MMS 

terminal cKent receives flie message, the presentation is parsed and rendered to display the 

interactive motoreycle promotion message, which looks like Figure 8. For example, the sentence 

"Attention A" shows a response message format When a userpushes the submit button, a 

response message is composed by MMS client application. The response message is as foflowing. 

<?xml versians'LO*' ©Qcoding^: 'lSO-8859-l"?> 
<moto> 

<moto>Buggy<^m(Mo> 

<modeI>Normal</model> 

<wheel> Normal <ywheel> 
</moto> 
</xml> 

In addition, the important feature of the application is the multimedia option menu, which 
shows the picture of tfie motors. The attention B in the sentences shows the presentation of 
multimedia intaactive controlleis. 

Another important feature of the application is link relationship between contreOer 
modules. In this exemplary embo diment description is given to the LINK relationship between 
controDers of model, picture and price. When the user chooses a model fiom (Select the model) 
option menu, the coiresponding picture and price of this model display at the same lime. The 
CONTAIN relationship is disclosed in the motor coflection and the models. If the coUection of a 
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certain motor is selected, afl models of the motorcycle collection can be seen in fte Mowing option 

mena AU other models, which do not belong to this motorcycle coUection, can not be seen. Tlie 

RXML and the multinKdia controller items arc shown in the Mowing XML document 

//the relational XML describe the relation between the controllas 
//the link relation 
//attention C 

<relation name='Wation 1" 
<rxn± link> 

<dtera altpigcfm; optionmenu ie&=*taodeP^ 

<item sXtrr^xfac oplionmenu modeI= *^ictui:tf ' re56=*^icture'!> 

<item attr=xfin: optionn^u rsS= *^ce"l> 
</cxml:link> 

<AxmL instant attmoro3aI> 

<6icml: instant > 
<Aelalion> 
//the contain relation 
//attention D 

<relalion name=*Mation 2'^ 
<rxml: contain> 

<itan atti^gtfm: optionmenu iefe*taoto'> 

<containitem attr^xfioa: oplionmKni rce='taoder^ 
<Axmt contain> 
<AtxmL' instant> 

-dtem atti^^cfin: item>Buggy</ item > 

< containitem atCP= xfin: item>Noimal</ containi tern > 

< containitem allP= xfrm it^ModeJ A</ containitem > 

< containitem altr= xfm: itemoModd B</ containitem > 
<AxmL' instant > 

<AeIation> 

According to an embodiment of the present invention a MMS service middleware is 
designed for MMS service providers to push MMS messages to users, and process incoming 
MMS mess^ for example fiom the MMS terminal, to provide value-added service. The sender 
of the message is authenticated by the authentication module, which uses the user management 
database to verify user. After the authentication, the message is parsed by a message par^. Then 
the message is passed on to the right message service for process via the service interfece. When a 

service initiates a message, the inessage is verified by a push trigger and 
interface. 
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Hk user intofeoe defines the size, fonn. position, show time, movement of interface 
parameters, and die inteifece defines how to add the data instances into the piesenlation of the 
pages. The user interface may be implemented wifli Ul-wares. An Ul-waic means a form, which 
can displayed and interactive in a predefined mode. See Rguie 1 1 for a le^^ 
shown can an independent item such as text, picture, video, audio, or a form 
button, dieck box, list box, radio button, option menu, input field. ^ connoUa- 
can be text, picture, video oraudia The forai controllers have the same interactive capabiHties as 
normal HIMLFormcontroUens or Xformsoontrolleis. In the implementation, the specification 
XFomis can be used here. Ite independent items have no r«tefined interactive capabm^^ 
interactive capability can be added in relations/actions part An interi^ace parameter can be shown in 
divcTK Ul-waies. For example, an inteifece parameter with multipltxlioice capability can be 

shown in a check box or a list box, because they are an riudtiple-choia contiolte 
important is. a data model can be rendered into diverse presentation for diverse MMS devices. 

The interactive capabilities of interfi«» pararneters describes whe^ 

dispky4)lay/edit/select. and wheflier it is of asii(gle^ice or multiple cto^^ Fbrexainple. a 

message can have two kinds of inierfius parameters. One is text, which can be m^^ 

Another is an audio array. User can select one item ftom the anay at a time and each of tiie it«^ 
canbeplayed. 

Up to now. the interactive approach for multimedia message, user temiinal and 
communication system have been described in details by means of exemplary embodiments. It 
should be understood that the present invention is not limited to die iUustraled embodiment 
examples, on die basis of the description of die embodimmts of die present invention, diose skilled 
in the art can make various modifications and improvements according to the present invention, 
without departure fiom the spirit and scope of the present invention. 
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CLAIMS 

I. An inteiactive method for multimedia message 

mullimedia message fem a server, generating a page dispb^^ 

message at an user terminal, wherein one or more controDeis are embedded into the displayed 
or played multimedia message and Ihe lelationship between said controllets is defined; and 
triggering at least one action event and perfiwming a coircsponding action in resfponse to the 
operations of the one or more controllers. 

2. The interactive melhod according to daiml, characterized in that the step of g^^ 
page fijr displaying or playing a multimedia message further includes: parang the received 
multimedia message to obtain flie pieseaitation structure of said multimedia message; and 
gaicrating a data model used in the page for displaying or playing said multimedia messages 
based on said multimedia message presentation structure. 

3. "Hie method acconiing to daiml, characterized in that said triggered action event is to send 
back a reqwnse message to the server which renders said multimedia message; and further 
includes a step of: in response to the operation of said oontiollers, automaticaUy genaating a 
response message for request to submit an Older. 

4. Themefliodaocordir^gtoclaiml.characteri2edinthatsaidtriggBreda^^^ 

the content of said multimedia message; and further includes a step of. in response to the 
operation of said controllers, displaying or playing said modified content of said multimedia 



5. Hie method according to any one of claims 1 to 4. characterized in that said relationship 
includes at least one of a LINK relationship and a COMTAIN relationship. 

6 . Hie method according to any one of claims 1 to 4, characterized inihat it further includes the 
steps of: parsing the recraved multimedia mess^ to obtain an action list of related action rules; 
listening in the triggered acti<» events based on said action list 

7. 'nKmethodaocordingtoanyoneofcIaimslto4,(±aracterizBdinthatsaidmultiiiiK^ 
message is coinposed in the laiiguage fotrns of XForms, XMU SMtt, XHTML or 

8. The method according to any one of claims 1 to 4. characterized in that said contioUers include 
one of a submit button, a selective button, a radio button, a check box, a text, a text field, a list 
box, an option mena 
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9. AusertenniMforamdiimediamess^seivice, adapted 

from a server and to process said multimedia message, said user temmal comprising: a device 
for receiving a multimedia message from a server, a device for geoeialing a pi«gp for displaying 
or playing said multimedia message; a device for embedding one or more controUos, between 
which relationships are defined, into said di^yed or played multimed^ 
for trigging at least one action event in response to an operation of said one or more 
controllers, and fijr implemeniing tlie coneqxMiding action. 

10. 'nwusertemiinaIaccQtdjiigtoclaim9,chanjcteti2edinthatthed^ 

for diqdjoong orplaying a innltimedia message ft^ a device for parang the 

received multimedia message to obtain the presentation structure of said multimedia message; 
and a device for generating a data model used in the page for displaying or playing said 
multimedia messages based on said multimedia message presentation stnicture. 

11. T^e user lEniiinal according to claim 9. characterized in that when said tt^^ 

to send back a response message to the server which lenders said multimedia message, in 
response to the operation of said controller, said means fijr triggering action event and for 
peiformirig coires^jonding action automatically generates a response message for request to 
submit an order. 

12. Theu8ertBmMnalaccotdir«toclaim9.characteri2edinthatwhensaid^^ 

to modify the content of said multimedia message, in response to the operation of said 
controller, said means for triggering action event and for performing corresponding action 
di^%s or plays said modified content of said multimedia message. 

13. The user terminal according to any one of claims 9 to 12, characterized in that said relationshq) 
includes at least one of a LINK relationship and a CONTAIN relationship. 

14. The user terminal according to any one ofclaims 9 to 12, characterized in that itfa^ 
includes a device for parsiiig the received multimedia niessage to 6^ 

action ndes; a device for listening in the triggered action events based on said ac^ 

15. Hie user tenninal according to any one of claims 9 to 12, characterized in that said multimedia 

message is composed in the language faniis of XFbmras, XIVIL, SMIU XHim or 
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16. TheusertBiminalaccoidiiigtoanyoneofdaiiiis9tol2,charact^^ 

includes one of a submit button, a selective button, a radio button, a chedc box, a text, a text 
field, a list box, an option menu. 

17. A commumcatioiisysm for inuliimedia message servk^ 
user tenniiials; one or inore sei^ sending a multinoed^ 
diaractaized in (hat a user terminal of said one OT inore use^ 

receiving a multimedia message fixMH a server, a device for generating a page for displaying or 
playing said multimedia message; a device for embedding one or more controllers, between 
which lelaiionships aie defined, into said displayed or played multimedia message; and a device 
fw trigging at least one action event in response to an operation of said one or mote 
controllas, and for perforaiing the conresponding action. 

18. A conqwtCT program for running on a user taminal within a multimedia communicaii 
wherein said communication system comprises one or more usa temiinals. and one or morB 
servers for sending a mdtimedia inessage to said user terminals, said computed 

on said user tenaainal for perfbraoiiig fifflowil^g steps: iBoavi^ 

server, generating a page for displayii« or playing said muWm^ 

more controllm are eoibedded into said displayed or played multimedia message and 

associations between said controllers are defined; and in response to an operation of said one 

or more controUeis. triggering at least one action event and performing the corresponding 

action. 
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